feat(rate-limiter): token bucket algorithm #2270

icecrasher321 · 2025-12-09T09:05:53Z

Summary

Improve system to use token bucket algo that prevents exploitation of boundary bursts and smoothens traffic better.

Type of Change

Other: Improvement

Testing

Testing Manually

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2025-12-09T09:05:58Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Preview	Comments	Updated (UTC)
docs	Skipped			Dec 9, 2025 10:50pm

greptile-apps · 2025-12-09T20:24:04Z

Greptile Overview

Greptile Summary

Replaces the fixed-window rate limiter with a token bucket algorithm to prevent exploitation of boundary bursts and smooth traffic better.

Key Changes:

Introduced RateLimitStorageAdapter interface with Redis and PostgreSQL implementations for token bucket state management
Each bucket tracks tokens (current available), lastRefillAt, and refills at refillRate per refillIntervalMs
Burst capacity is 2x the sustained rate (e.g., 10 req/min → 20 max burst tokens)
Database migration drops old user_rate_limits table and creates new rate_limit_bucket table
Fails closed on storage errors (denies requests rather than allowing them)
Updated API documentation to explain token bucket behavior with maxBurst field

Previous feedback addressed:

Fixed logic issue in db-token-bucket.ts (simplified to tokens >= 0)
Changed deprecated HMSET to HSET in Redis Lua script

Confidence Score: 4/5

This PR is safe to merge after verifying the migration path for existing rate limit data.
Well-designed token bucket implementation with clean abstractions, comprehensive tests, and proper documentation. Previous review feedback has been addressed. Score of 4 (not 5) due to the destructive migration that resets all existing rate limit state.
Pay attention to packages/db/migrations/0119_far_lethal_legion.sql - the migration drops the old rate limit table, which will reset all users' rate limit counters upon deployment.

Important Files Changed

File Analysis

Filename	Score	Overview
apps/sim/lib/core/rate-limiter/rate-limiter.ts	5/5	Complete refactor from fixed-window to token bucket algorithm. Clean abstraction using storage adapters with proper error handling (fail closed on storage errors).
apps/sim/lib/core/rate-limiter/types.ts	5/5	Config updated to use TokenBucketConfig with maxTokens (2x multiplier for burst), refillRate, and refillIntervalMs parameters.
apps/sim/lib/core/rate-limiter/storage/db-token-bucket.ts	4/5	PostgreSQL token bucket implementation using atomic INSERT...ON CONFLICT DO UPDATE. Complex SQL for token refill calculations executed in single query.
apps/sim/lib/core/rate-limiter/storage/redis-token-bucket.ts	5/5	Redis token bucket using Lua scripts for atomic token consume/status operations. Uses HSET (fixed from deprecated HMSET).
apps/sim/lib/core/rate-limiter/rate-limiter.test.ts	5/5	Comprehensive tests for token bucket rate limiter covering all trigger types, subscription plans, storage errors, and bucket reset functionality.
packages/db/migrations/0119_far_lethal_legion.sql	4/5	Creates new rate_limit_bucket table and drops old user_rate_limits table. Destructive migration - existing rate limit state will be reset.

Sequence Diagram

sequenceDiagram
    participant Client
    participant RateLimiter
    participant StorageFactory
    participant Redis/DB
    
    Client->>RateLimiter: checkRateLimitWithSubscription()
    RateLimiter->>RateLimiter: getBucketConfig(plan, counterType)
    RateLimiter->>StorageFactory: createStorageAdapter()
    StorageFactory-->>RateLimiter: Redis or DB adapter
    
    RateLimiter->>Redis/DB: consumeTokens(key, 1, config)
    
    Note over Redis/DB: Atomic operation:<br/>1. Calculate elapsed time<br/>2. Refill tokens (capped at maxTokens)<br/>3. Consume if tokens >= requested<br/>4. Return new state
    
    Redis/DB-->>RateLimiter: ConsumeResult {allowed, tokensRemaining, resetAt}
    
    alt Allowed
        RateLimiter-->>Client: {allowed: true, remaining, resetAt}
    else Rate Limited
        RateLimiter-->>Client: {allowed: false, remaining: 0, retryAfterMs}
    end

greptile-apps

_{19 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

apps/sim/lib/core/rate-limiter/storage/redis-token-bucket.ts

apps/sim/lib/core/rate-limiter/storage/db-token-bucket.ts

icecrasher321 · 2025-12-09T22:50:48Z

@greptile

greptile-apps

_{19 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

… docs, mcp, autolayout improvements (#2286) * fix(mcp): prevent redundant MCP server discovery calls at runtime, use cached tool schema instead (#2273) * fix(mcp): prevent redundant MCP server discovery calls at runtime, use cached tool schema instead * added backfill, added loading state for tools in settings > mcp * fix tool inp * feat(rate-limiter): token bucket algorithm (#2270) * fix(ratelimit): make deployed chat rate limited * improvement(rate-limiter): use token bucket algo * update docs * fix * fix type * fix db rate limiter * address greptile comments * feat(i18n): update translations (#2275) Co-authored-by: icecrasher321 <icecrasher321@users.noreply.github.com> * fix(tools): updated kalshi and polymarket tools to accurately reflect outputs (#2274) * feat(i18n): update translations (#2276) Co-authored-by: waleedlatif1 <waleedlatif1@users.noreply.github.com> * fix(autolayout): align by handle (#2277) * fix(autolayout): align by handle * use shared constants everywhere * cleanup * fix(copilot): fix custom tools (#2278) * Fix title custom tool * Checkpoitn (broken) * Fix custom tool flash * Edit workflow returns null fix * Works * Fix lint * fix(ime): prevent form submission during IME composition steps (#2279) * fix(ui): prevent form submission during IME composition steps * chore(gitignore): add IntelliJ IDE files to .gitignore --------- Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: waleedlatif1 <waleedlatif1@users.noreply.github.com> * feat(ui): logs, kb, emcn (#2207) * feat(kb): emcn alignment; sidebar: popover primary; settings-modal: expand * feat: EMCN breadcrumb; improvement(KB): UI * fix: hydration error * improvement(KB): UI * feat: emcn modal sizing, KB tags; refactor: deleted old sidebar * feat(logs): UI * fix: add documents modal name * feat: logs, emcn, cursorrules; refactor: logs * feat: dashboard * feat: notifications; improvement: logs details * fixed random rectangle on canvas * fixed the name of the file to align * fix build --------- Co-authored-by: waleed <walif6@gmail.com> * fix(creds): glitch allowing multiple credentials in an integration (#2282) * improvement: custom tools modal, logs-details (#2283) * fix(docs): fix copy page button and header hook (#2284) * improvement(chat): add the ability to download files from the deployed chat (#2280) * added teams download and chat download file * Removed comments * removed comments * component structure and download all * removed comments * cleanup code * fix empty files case * small fix * fix(container): resize heuristic improvement (#2285) * estimate block height for resize based on subblocks * fix hydration error * make more conservative --------- Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: icecrasher321 <icecrasher321@users.noreply.github.com> Co-authored-by: waleedlatif1 <waleedlatif1@users.noreply.github.com> Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com> Co-authored-by: mosa <mosaxiv@gmail.com> Co-authored-by: Emir Karabeg <78010029+emir-karabeg@users.noreply.github.com> Co-authored-by: Adam Gough <77861281+aadamgough@users.noreply.github.com>

icecrasher321 added 2 commits December 8, 2025 23:20

fix(ratelimit): make deployed chat rate limited

cf2b9f1

improvement(rate-limiter): use token bucket algo

1efe8d5

update docs

44e356f

vercel bot deployed to Preview December 9, 2025 19:27 View deployment

fix

3cf32bb

vercel bot temporarily deployed to Preview December 9, 2025 20:03 Inactive

fix type

73f3ae4

vercel bot temporarily deployed to Preview December 9, 2025 20:13 Inactive

icecrasher321 marked this pull request as ready for review December 9, 2025 20:14

greptile-apps bot reviewed Dec 9, 2025

View reviewed changes

apps/sim/lib/core/rate-limiter/storage/redis-token-bucket.ts Outdated Show resolved Hide resolved

apps/sim/lib/core/rate-limiter/storage/db-token-bucket.ts Outdated Show resolved Hide resolved

apps/sim/lib/core/rate-limiter/storage/db-token-bucket.ts Show resolved Hide resolved

fix db rate limiter

17dbfbb

vercel bot temporarily deployed to Preview December 9, 2025 22:44 Inactive

address greptile comments

169c348

vercel bot temporarily deployed to Preview December 9, 2025 22:50 Inactive

greptile-apps bot reviewed Dec 9, 2025

View reviewed changes

icecrasher321 merged commit aea32d4 into staging Dec 9, 2025
9 checks passed

waleedlatif1 mentioned this pull request Dec 9, 2025

feat(i18n): update translations #2275

Merged

10 tasks

waleedlatif1 deleted the feat/token-bucket-algo branch December 10, 2025 04:03

waleedlatif1 mentioned this pull request Dec 10, 2025

v0.5.23: kb, logs, general ui improvements, token bucket rate limits, docs, mcp, autolayout improvements #2286

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(rate-limiter): token bucket algorithm #2270

feat(rate-limiter): token bucket algorithm #2270

Uh oh!

icecrasher321 commented Dec 9, 2025 •

edited

Loading

Uh oh!

vercel bot commented Dec 9, 2025 •

edited

Loading

Uh oh!

greptile-apps bot commented Dec 9, 2025 •

edited

Loading

Uh oh!

greptile-apps bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

icecrasher321 commented Dec 9, 2025

Uh oh!

greptile-apps bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat(rate-limiter): token bucket algorithm #2270

feat(rate-limiter): token bucket algorithm #2270

Uh oh!

Conversation

icecrasher321 commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel bot commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps bot commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Overview

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

icecrasher321 commented Dec 9, 2025

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

icecrasher321 commented Dec 9, 2025 •

edited

Loading

vercel bot commented Dec 9, 2025 •

edited

Loading

greptile-apps bot commented Dec 9, 2025 •

edited

Loading